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I. (U) PURP OSE; 

(U) Purpose of this report is to review initial STAR GATE 
procedures and to provide recommendations for revisions based on 
recent experience including DESERT STORM tasking. 

II. (U) SCOPE ; 

(S/NF) This report examines data development, evaluation and 
assessment procedures for data generated by STAR GATE sources. 

III. (U) SUMMA RY; 

A. (S/NF) Lessons learned -from recent DESERT STORM tasking have 
led to several .immediate changes in STAR GATE procedures as discussed 
in Section IV. Some recommendations -for -future revisions are also 
identified. 

B. (S/NF) Evaluations of future STAR GATE reporting will be 

prepared in accordance with the criteria and provisions of DIAM 58-13, 
Subject: The Department of Defense HUMINT Management System, dated 28 

March 1988. 

IV. (U) BACKGROUND ; 

A. (S/NF) In December 1990 and January 1991, initial efforts 

were made to provide an effective and viable system to evaluate STAR 
GATE reporting. These procedures are illustrated in Figure 1. The 
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initial proposals resulted in the implementation of procedures that 
attempted to assess both the utility and the value of STAR GATE 
reporting . 

B. (S/NF) During the course of an off-site conference held on 7 

* and 8 March 1991 at Fort Dietrich, senior analysts from DT-5A, DT-3B, 

and DT-S, identified shortfalls in current evaluation procedures and 
prescribed revisions that would allow for the implementation of an 
objective and workable system. The senior analysts believed that 
assessments relating to the accuracy of STAR GATE reporting could 
best be addressed by applying applicable scientific methodologies to 
STAR GATE material. Procedures already exist and can be implemented 
available project ADF' methods. Specific accuracy assessments would be 
determined by intelligence analysts in conjunction with appropriate 
outside scientific review. The same analysts were of the opinion that 
the intelligence value or utility of STAR GATE reporting could best be 
determined by select intelligence analysts directly involved with 
specific task areas. 

C. (U) What follows are the results of a comprehensive group 
effort conducted to identify procedural shortfalls and recommend the 
implementation of the following revisions. 

V. (S/NF/SG/LIMDI5) SHORTFALLS AND REVISIONS ; 

^ - SHORTFA LL; Need to identify action officers with whom DT-3B 

can deal directly to expedite the evaluation process and insure that 
such evaluations are accomplished in the most successful manner. 

GATE / 
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analytical evaluation process. Direct involvement with these (and 
other) scientific analysts should help in better task definition and 
in identifying ways for improving potential utility. 

SHORTFALL ; Procedures to date appear burdensome and need 
streamlining. 

“ REVISI ON; Procedures developed by the R&D community should be 
used for assessing the accuracy of project reporting. The 
intelligence value of STAR GATE reporting must be determined by 
intelligence community analysts working from an all source 
intelligence perspective and environment. This should be based on an 
agreed-upon definition of utility and what target data would be 
required to satisfy the utility criteria. 

SHORTFALL ; Need to assess whether our quantifiable answer 
categories would be expanded to reflect the range of uncertainty when 
one is evaluating a DT-S report. 

- REVISION ; Specificity in the judicious selection of mission 
requirements will go a long way towards streamlining the process. 
Target selection must be based on targets against which STAR GATE has 

NO FORN/ L I MD I S 
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a -favorable track record. To this extent, limits to STAR BATE data 
(or project type) , need to be provided in advance to the tasking 
el ements. 

SHORTFALL; Need to re-accompl i sh certain original tasking, 
updating it as necessary to the current situation. 

REVISION; Tasking should be levied within specific time 
windows and not be left open-ended. Requirement originators should be 
reminded of their responsibility to update requirements and changes of 
collection emphasis. 

SHORT FALL ; Senior analysts need to insure that the 
intelligence evaluations (ie, utility assessments) of project 
reporting are timely, objective, substantive and prepared in 
consonance with established intelligence community procedures. 

- REVISI ON; STAR GATE reporting needs to be evaluated in 
consonance with the DIA HUMINT evaluation scheme paraphrased 

substantially as follows; 

RATING VALUE 

Of Major Significance Having Impact on National 

Policy and Decision-Making 
Of High Value Best Effort - First Report 

in a Field of Endeavor 

Of Value Supplements - Updates - 

Confirms Known Data 

FORN/ L I MD I S 
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Of Low Value Poor Quality - Not Timely 

□f No Value Erroneous - Misleading 

There is another consideration. Proper assessment procedures •for 
either accuracy or utility require that criteria for evaluation be 
specific in advance. Consequently, it should be appropriate for a 
candidate user (or task originator) to clearly define what type of 
data would have potential utility, and how it would be used if 
provided and how it would be evaluated. This type of data could 
include specific essential elements of information (EEI) and required 
timing. A troublesome problem may be encountered if the specific EEI 
(if posed as a question) provides too much background information for 
the source that could adversely influence session results. To avoid 
such difficulty, it may be necessary to conduct the session in several 
phases, time permitting. This issue is not easy to resolve and 

remains as a future action item. <A basic review of this overall 
issue can be found in DT-S-1010-B, 13 Dec 1990, Evaluation Methods). 

SHORTFALL; Need to reconsider acceptance of tasking based on 
availability and timeliness of known ground truth. 

“ REVISION ; The STAR SATE collection management system must be 
selective in its acceptance of requirements for which there is a 
likelihood of learning ground truth within a reasonable amount of 
time. A procedure needs to be established that allows for rapid 
follow-up for task refinement. 
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SHORTF ALL : Multiple source reporting in a combined summary 
•format is confusing precludes affixing responsibility to the 
reporting. 

R EVISION : The principle of "one source one report" must be 
strictly adhered to so that reporting may be attributable to a 
specific source. Consequently, data from any source will be reported 
separately or in easily identifiable and separated sections of an 
overall summary report. 

SHORTF ALL: Current STAR GATE reporting is inconsistent with 
other established HUMINT reporting formats. 

- REVISION : Reports need to be prepared in accordance with the 
criteria used in the preparation of Intelligence Information Reports 
(IIR's). Each report will bear a numerical source designator that 
will protect the identity of the source. An example of the reporting 
format is at Figure 2. A Summary of Information (SOI) will be 
prepared by the source. It will reflect a project number , source 
number, session number, and date of report. The SOI will be attached 
as an addendum to the formal report. An example of an SOI is at 
Figure 3. 


- SHORTFALL : STAR GATE predictions are rarely accurate. 

REVISION: Predictive reporting is indeed the project's 

weakest feature. The results of searches are conducted at a slightly 
higher rate of success. Hi stor i cal ly , the project s track record 

GATE / 
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reveals a relatively high rate of success when mission tasking is 
targeted against fixed, static targets located in past or current time 
windows. Consequently, project tasking should be prioritised 
according to estimated success probability. 

~ SHORTFALL S STAR GATE project reporting should not stand alone. 

R EVISION S Senior analysts need to insure that STAR GATE is 
an integral part of the all -source collection management system, a key 
tip-off player acting in concert with other intelligence disciplines. 

SHORTFALL S Highlights of singularly outstanding reporting 
should be recorded and made readily retrievable. 

- REVISION S A "Kudo File" will be established to facilitate the 
retrieval of examples of high quality reporting. Factors that may 
have contributed to these results will be identified whenever 
possible. 
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DATA DEVELOPMENT, EVALUATION AND ASSESSMENT 

REQUIREMENTS i 
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PROJECT STAR GATE 




PROJECT NUMBER :: 
DATE OF SESSION:: 
SESSION STARTs 
METHOD :: 


SOURCE NUMBER'S 
DATE OF REPORTS 
SESSION ENDS 
SESSION NUMBER:: 


( S / IMF ) BACKGROUND s 


2. (S/IMF) ORIGINATOR REQUIREMENT'S 


3. (S/IMF) SOURCE TASK: 

4. (S/NF) OPS OFFICER COMMENTS: Address inclemencies. 

Include any info not contained in Summary o-f information (SOI) 
p r o v i d e d by so u r c e 

5. (S/IMF) EVALUATION: Enter numerical designator IAW DIAM 58- 

13. 

6. (S/NF) SEARCH EVALUATIONS 


Not Releasable to foreign nationals 
L. i m i t e cl d i s t rib u tion 


CLASSIFIED BY: DIA/DT 

DECLASSIFY ONs OADR 


dmmmm imoforn / 1. 1 md i s 


APPENDIX ^ Figure 2 


Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9 



Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9 

EXAMPLE 


PROJECT: 
SOURCE ! 
DATE: 

sess :i: ON : 


SUMMARY OF INFORMATION 


T HE SOI IS ATTACHED TO THE REPORT 

SKETCHES PREPARED BY T H E SOURCE I N 

sol 


SUPPORT OF FINDINGS 


FOLLOW 




* 


APPENDIX C| Figure 3 
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